TSAO, Doris, FACE PROCESSING Mechanisms, Caltech, Apr 11 2013

This is in Clark Aud at Stanford Med Center at 4:30PM on Thursday.

her mentor was David Hubel.

she opens with quotes from Hubel and Wiesel in 59 and 65:
what are the transformations... (that visual info goes thru)  at ever higher levels.

side view of macaque brain

retina to LGN to V1  and  ultimately to IT = inferior temporal cortex.

overarching goal:  how does cortex integrate elementary features to recognize complex shapes
that are improtant for ethological behavior..

each part of V1 is doing the same thing but at difft parts of visual space.

Charlie Gross:  see Bruce et al in 1981... discovered face selective neurons

1997:  Nancy Kanwisher, McDermott and Chun in  J Neurosci
Nancy called it the FFA = fusiform face area  (using  BOLD/ fMRI)

within a single voxel in humans:  10s of thousands of neurons...

Doris studies monkeys in her lab  (macaques)

awake and head-fixed in fMRI...

she has found  6 areas of  face selective cortex:  (bilat symmetric)

located in the same place across animals and in both hemispheres.

start in middle face patch (still in monkey)

present each image  5 to 10 times

she shows a movie showing the stimuli that  the monkey saw and we listen to clicks from a single neuron:

neuron seems obviously sensitive to faces  (to all the different faces in the stim set)

almost of the cells in this region were face selective:

182 of 241 neurons  :

Summary

6 face-selective patches in monkey...

fMRI is highly correlated to single unit activity in this experiment  = expt.

now,  how does it work?...

work by Sebastian Moeller
see Moeller in Science 2008;

they stimulate  intracortically to see what lights up  (while monkey is looking at a blank screen)

connections of ML (one of the face selective regions)

shows that ML is connected to all the other (five) face patches...

same with ALL the other face patches  ie  they all form a network
among themselves but NOT to the neighboring cortex;

what are the inputs to the face patch system?

work of Piercesare Grimaldi...

inject tracers into face patch regions...

used Fast Blue... retrograde tracer...

PL is the most posterior patch...

we found strong labeling to V4  (ie major  input comes from V4)

also  saw labeling in Claustrum  (an enigmatic brain region (that Francis and Christof were fond of)

also saw labeling in ML and in AM  (the most anterior face patch)

also to all the other face patches:  and to OFC = orbitofrontal cortex  (faces have high social value)

big inputs from ventral claustrum to ML and PL  and inputs from the Pulvinar to both.

so, we suspect that pulvinar and claustrum are also doing face processing;

face patches have very low connectivity to other areas

she shows the overall circuit diagram

slide:  Anatomical tracer summary

(diagram that includes all the above connections shown by their tracer studies)

***************
Now how does it work?
what is the neural mechanism for face detection

Local Contrast Cues

state of the art  artificial face deteciton systems use thousands of simple classifiers
 that make a decision based on local contrast cues.
(cameras use this)

Pawan Sinha  suggests  in 2002  that
faces should be invariant to contrast and illumination;

Are cells in the middle face patches using contrast features?

Ohayon in Neuron 2012

they use different facial parts to see which parts drive the cell

try pairs of face parts with differing part intensity / contrast

eg  forehead and L eye;

for each cell can compute a significant feature histogram...

compare various pairs incorporating  forehead, nose, L eye, chin etc.

slide:  # of cells that  were  tuned for a particular  pair...
eg 100 cells prefer  Left eye darker than the nose

consistent across monkeys...

=>  local contrast is very important

Summary:  these results  show the importance of contrast in generating face selective responses...

(looking at cells in ML and MF)

Surya comment:  why not  use a (combined) linear feature consisting of  11 facial features;

her comment:  different cells are sensitive to different combos of features.

What is the contribution of SHAPE to face detection?

to study that question they use parametrized Cartoon faces:

cells respond similarly to real face and to cartoon

implication: can use cartoons to study cell tuning;

look at 2**7  different stimuli  (consisting of  hair on or off/ nose on or off etc.)
outline, pupils, mouth;  

what matters is ecological meaning  (ie faces are important)

How do we do individual face identities?

return to cartoon faces to answer that:

19 different dimensions  each with 11 different values
eg aspect ratio of face;  intra-eye distance;  hair length;

results:  indiv cells are tuned to these various features

one partic cell:  face aspect ratio;  inter-eye distance,  iris size;  etc

she shows facial caricatures:  cartoons

she shows a movie:  cartoon face with different eye sizes;
also shows fixation point of monkey

(is the cell just responding to that particular feature: eg a nose?  NO!
when you take away the rest of the face; the cell stops firing.)

so feature are not generally tuned in isolation...

eg Burt and Ernie face;  these cells see the whole face
(not just a sum of features)

*************
how does it work?

take a face in the background and superimpose on it a noise pattern to
see which noise patterns reinforce firing of a partic cell.
(she shows a faint face with a cloud of noise superimposed: that may emphasize
the dark hair or dark eyes, etc)

slide:  result from another MF cell:
noise that excites cells  are ones that have dark where the hair is, eg...

what is the specialization of more anterior regions in the face processing network; AL and AM
(two anterior patches)

Freiwald and Tsao in Science  (2010 ?)

look at head orientations;

what is AL doing:  soln to puzzle:

look at  25 different people all looking to the left

a new property of AL  (mirror symmetry; profile selective)

she shows movie showing clearly preference for this cell for profiles that are turned eith to the left or right;

(that was in AL)

now AM:  (quite difft than AL);

AL responds only to single individual, regardless of where he/she is looking...

monkey did not know that person (for which the cell was firing selectively)

(regardless of head orientation)

AM also distinguised monkey faces from human faces...

slide:  population correlation matrix:

AL:  view invariant identity tuning  (within single individuals;
 ie  monkey looking only at one person's face
regardless of orientation of head)

Summary:

AM :  many cells sparsely tuned to  specific faces

hierarchy of facial processing  going from ML to AM to AL

****

behavioral evidence for the importance of involvement of these patches;

they studied Maurice, a partic  monkey that had only 3 face patches; (as opposed to the expected 6)

Maurice  did object ID just as well as the other monkeys...

BUT  Maurice was terrible at face id ...

Task:  report if 2 faces are same or difft in identity...

see face for only 500 ms;

if faces were identical... then monkeys did well above chance
now same indiv but  with difft expression (neutral vs surprised, eg)

but now stimulate ML during this task...  then performance goes way down  (

(also  try this with microstim of AM)  huge  decrease in recognition ability;

and highly specific to faces  (try this with objects and no decrease in performance)

Now what might the monkey be experiencing

(she shows Parvizi and Kalanit GS clip)

(this is a great  video clip... one of my favorites;  epileptic pt  who is undergoing  ablation)

***
Big theoretical debate among researchers: 
are non-maximal  responses used by the brain?

eg when viewing clocks or apples  that may look kind of like faces but not really.

*************

Macaque Face Processing : '6 face patches in IT cortex, dedicated to face proc.

patches are strongly interconnected...

Next step (in the research program):  record from identified projection neurons

(among the 6 face patches)

use optogenetics and antidromic stim;

Scene Processing:

PPA:  parahippocampal place area: (is involved in scene processing)

she shows a movie of a cell recorded from LPP:

responds to familiar scenes;

returning to the beginning of her talk:
returns to H and W 1965 quote  (abt cells specifically tuned)

acknowledgements;  many funders:  

Q  do face patches obey cytoarchitectonic boundaries?
A  NO

Bill N:  your work seems to show that fMRI is a good predictor of single unit physio.
Have you ever seen the BOLD steer you wrong?

A:  fMRI only provides the initial clue as to where to look;
 BOLD is no good at distinguishing differences in face patches  or individual identities;
we fail miserably;  fmRI is good for localizing  face patches but not for really characterizing their tuning.

comment:  Jim DiCarlo work in PL:  really posterior area that just responds to  single face parts;

Pat S:  how are familiar face  parts combined to distinguish  one partic  familar face from another

A:  could be something like Hebbian Learning.  (learning a partic. combo of sparsely tuned neurons)

q  (Kalanit):  what is hierarchic  relationship among these patches?
A:  have not found a clear  hierarch relationship
more complex that  simple feedforward;  lots of feedback;
we do not have that picture yet...